Review of the techniques used in motor‐cognitive human‐robot skill transfer

Abstract A conventional robot programming method extensively limits the reusability of skills in the developmental aspect. Engineers programme a robot in a targeted manner for the realisation of predefined skills. The low reusability of general‐purpose robot skills is mainly reflected in inability in novel and complex scenarios. Skill transfer aims to transfer human skills to general‐purpose manipulators or mobile robots to replicate human‐like behaviours. Skill transfer methods that are commonly used at present, such as learning from demonstrated (LfD) or imitation learning, endow the robot with the expert's low‐level motor and high‐level decision‐making ability, so that skills can be reproduced and generalised according to perceived context. The improvement of robot cognition usually relates to an improvement in the autonomous high‐level decision‐making ability. Based on the idea of establishing a generic or specialised robot skill library, robots are expected to autonomously reason about the needs for using skills and plan compound movements according to sensory input. In recent years, in this area, many successful studies have demonstrated their effectiveness. Herein, a detailed review is provided on the transferring techniques of skills, applications, advancements, and limitations, especially in the LfD. Future research directions are also suggested

A computational model of human table tennis for robot application

Table tennis is a difficult motor skill which requires all basic components of a general motor skill learning system. In order to get a step closer to such a generic approach to the automatic acquisition and refinement of table tennis, we study table tennis from a human motor control point of view. We make use of the basic models of discrete human movement phases, virtual hitting points, and the operational timing hypothesis. Using these components, we create a computational model which is aimed at reproducing human-like behavior. We verify the functionality of this model in a physically realistic simulation of a BarrettWAM

Reinforcement Learning by Relative Entropy Policy Search

Policy search is a successful approach to reinforcement learning. However, policy improvements often result in the loss of information. Hence, it has been marred by premature convergence and implausible solutions. As first suggested in the context of covariant policy gradients, many of these problems may be addressed by constraining the information loss. In this book chapter, we continue this path of reasoning and suggest the Relative Entropy Policy Search (REPS) method. The resulting method differs significantly from previous policy gradient approaches and yields an exact update step. It works well on typical reinforcement learning benchmark problems. We will also present a real-world applications where a robot employs REPS to learn how to return balls in a game of table tennis

Learning Table Tennis with a Mixture of Motor Primitives

Table tennis is a sufficiently complex motor task for studying complete skill learning systems. It consists of several elementary motions and requires fast movements, accurate control, and online adaptation. To represent the elementary movements needed for robot table tennis, we rely on dynamic systems motor primitives (DMP). While such DMPs have been successfully used for learning a variety of simple motor tasks, they only represent single elementary actions. In order to select and generalize among different striking movements, we present a new approach, called Mixture of Motor Primitives that uses a gating network to activate appropriate motor primitives. The resulting policy enables us to select among the appropriate motor primitives as well as to generalize between them. In order to obtain a fully learned robot table tennis setup, we also address the problem of predicting the necessary context information, i.e., the hitting point in time and space where we want to hit the ball. We show that the resulting setup was capable of playing rudimentary table tennis using an anthropomorphic robot arm

Relative Entropy Policy Search

Policy search is a successful approach to reinforcement learning. However, policy improvements often result in the loss of information. Hence, it has been marred by premature convergence and implausible solutions. As first suggested in the context of covariant policy gradients (Bagnell and Schneider 2003), many of these problems may be addressed by constraining the information loss. In this paper, we continue this path of reasoning and suggest the Relative Entropy Policy Search (REPS) method. The resulting method differs significantly from previous policy gradient approaches and yields an exact update step. It works well on typical reinforcement learning benchmark problems

Reinforcement Learning by Relative Entropy Policy Search

Policy search is a successful approach to reinforcement learning. However, policy improvements often result in the loss of information. Hence, it has been marred by premature convergence and implausible solutions. As first suggested in the context of covariant policy gradients, many of these problems may be addressed by constraining the information loss. In this book chapter, we continue this path of reasoning and suggest the Relative Entropy Policy Search (REPS) method. The resulting method differs significantly from previous policy gradient approaches and yields an exact update step. It works well on typical reinforcement learning benchmark problems. We will also present a real-world applications where a robot employs REPS to learn how to return balls in a game of table tennis

Cyclic tensile tests of Shetland pony superficial digital flexor tendons (SDFTs) with an optimized cryo-clamp combined with biplanar high-speed fluoroscopy

Background: Long-term cyclic tensile testing with equine palmar/plantar tendons have not yet been performed due to problems in fixing equine tendons securely and loading them cyclically. It is well established that the biomechanical response of tendons varies during cyclic loading over time. The aim of this study was to develop a clamping device that enables repetitive cyclic tensile testing of equine superficial digital flexor tendon for at least 60 loading cycles and for 5 min. Results: A novel cryo-clamp was developed and built. Healthy and collagenase-treated pony SDFTs were mounted in the custom-made cryo-clamp for the proximal tendon end and a special clamping device for the short pastern bone (os coronale). Simultaneously with tensile testing, we used a biplanar high-speed fluoroscopy system (FluoKin) to track tendon movement. The FluoKin system was additionally validated in precision measurements. During the cyclic tensile tests of the SDFTs, the average maximal force measured was 325 N and 953 N for a length variation of 2 and 4 % respectively. The resulting stress averaged 16 MPa and 48 MPa respectively, while the modulus of elasticity was 828 MPa and 1212 MPa respectively. Length variation of the metacarpal region was, on average, 4.87 % higher after incubation with collagenase. The precision of the FluoKin tracking was 0.0377 mm, defined as the standard deviation of pairwise intermarker distances embedded in rigid bodies. The systems accuracy was 0.0287 mm, which is the difference between the machined and mean measured distance. Conclusion: In this study, a good performing clamping technique for equine tendons under repetitive cyclic loading conditions is described. The presented cryo-clamps were tested up to 50 min duration and up to the machine maximal capacity of 10 kN. With the possibility of repetitive loading a stabilization of the time-force-curve and changes of hysteresis and creep became obvious after a dozen cycles, which underlines the necessity of repetitive cyclical testing. Furthermore, biplanar high-speed fluoroscopy seems an appropriate and highly precise measurement tool for analysis of tendon behaviour under repetitive load in equine SDFTs